Evaluation criteria for health apps supporting medication adherence in early-stage technology development - a scoping review.

Introduction: Health apps offer an approach to improve the patients' management of their medication. Although the Digital Healthcare Act (DVG) has created a claim in the statutory health insurance (SHI), the large number of health apps available and their varying quality make it difficult for service providers and especially for medical laypersons to select an adequate high-quality medication app. Manufacturers need guidance for the development of high-quality apps right from the start. Various general evaluation concepts for health apps have been available to date. However, the requirements that should be met by healthcare depend largely on the field of application and the type of apps. This article aims to provide an overview of the international evidence on specific criteria for the evaluation of medication apps. Methods: Within the framework of a scoping review, a systematic search was conducted in PubMed and EMBASE on January 29, 2020. The search was limited to publications from 2007 onwards as well as to English and German articles. Additionally, a semi-systematic research of reference lists of the previously included articles as well as a structured search of websites of relevant stakeholders were conducted. Inclusion criteria were the following: the publication deals with health apps that can be used on smartphones and focus on supporting medication intake; the publication does not refer to evaluation criteria for a single app exclusively. The included publications were examined in a qualitative content analysis searching for evaluation criteria and categorizing them according to the framework criteria of the DVG and the Digital Health Applications Ordinance (DiGAV). Results: 2,542 articles were identified in the systematic search (999 in PubMed, 1,543 in EMBASE, 560 duplicates). A total of 16 studies met the inclusion criteria. The semi-systematic research and the structured search identified one further study. A catalog of criteria was developed based on the included 17 studies. This catalog covers the general topics "patient orientation" (data protection and security, consumer protection, user friendliness) and "quality/core functions of medication apps" (reminder, self-monitoring, (drug) information, motivation to change behavior, drug/patient safety, robustness) as well as "interoperability/cooperation". Due to its specific importance for medication apps, the subcategory "motivation for behavioral change" stands out beneath the general topic "quality/core functions of medication apps". This category aims to evaluate the design of individual functions with regard to their potential to actually change the behavior of app users. Discussion: The criteria for the evaluation of health apps mentioned in the DiGAV intersected with the criteria identified in the literature research. However, the area of positive health care effects was hardly covered by the included studies. In the development of the criteria catalog, it was not possible to weight the identified criteria. Therefore, the catalog should be understood as a supporting checklist for service providers, manufacturers, and/or users. Conclusions: A large variety of possible evaluation criteria for medication apps could be shown. Future research should focus on the possibilities of weighting these diverse evaluation criteria, using not only clinical studies but also methods to identify preferences.

Occupational therapy for persons with cognitive impairments.

Background: Damage to the central nervous system can occur in adulthood, for example, due to stroke, trauma, tumours, or chronic diseases. After damage to the central nervous system, cognitive impairments occur in addition to physical limitations. Occupational therapy is most often prescribed for neurological diagnoses, including stroke and traumatic brain injury. Methods: The health technology assessment (HTA) report this HTA article is based on investigates the clinical effectiveness, cost-effectiveness, and patient-related, social and ethical aspects of occupational therapy for patients with cognitive impairments compared to no occupational therapy. In addition, the effects of different occupational therapy interventions with and without cognitive components were compared in an explorative overview. Patients with moderate or severe dementia are excluded from the assessment. Systematic overviews, that is, systematic reviews of systematic reviews, were conducted. Results: For the evaluation of clinical effectiveness, a total of nine systematic reviews were included. No systematic review was identified for the assessment of costs or cost-effectiveness. Five systematic reviews were included for the assessment of patient and social aspects. For the assessment of clinical effectiveness compared with no occupational therapy, five systematic reviews comprising 20 randomised controlled trials with a total of 1,316 subjects reported small positive effects for the outcomes "global cognitive function" and "activities of daily living" as well as a non-quantified positive effect on the outcomes "health-related quality of life" and "behavioural control". No effect was found for individual components of cognition and measures of perception. The quality of the evidence for all outcomes is low due to a high risk of bias. In the supplementary presentations, no positive effects could be demonstrated on the basis of the available evidence. The quality of this evidence was not assessed. For the assessment of patient and social aspects, five systematic reviews on patients with a stroke or a traumatic brain injury - without specification regarding cognitive deficits or studies with their relatives - were included. It was reported that patients and family caregivers go through different phases of rehabilitation in which the discharge home is a decisive turning point. The discharge home represents a crucial breaking point. Regaining an active, self-determining role is a process that requires therapists to find the right level of support for patients and relatives. For the assessment of ethical aspects, nine documents were included. We identified ethical problem-solving models for occupational therapy and 16 ethical aspects in occupational therapy for cognitive deficits. The central theme of the analysis is the limited autonomy due to the consequences of the disease as well as the resulting tensions with those treating the patient. Conclusions: Based on this systematic overview, it can neither be proven nor excluded with certainty that occupational therapy for cognitive impairment is an effective therapy for adult patients with central nervous system injuries compared to no occupational therapy. There is a lack of randomised trials with sufficient sample size, well-defined interventions, and comparable concomitant therapies in the control groups, but there is also a lack of well-designed observational studies in routine care and health economic studies. The identified systematic reviews on patient and social aspects provide information on the needs of patients after stroke or traumatic brain injury and their relatives, but there is a lack of studies on this aspect in German-speaking countries. For the ethical assessment, in addition to the identified theoretical models for solving ethical conflicts in occupational therapy, more empirical studies on ethical aspects with patients with cognitive deficits and their relatives as well as occupational therapists are needed.

Patient' and social aspects related to complementary mistletoe therapy in patients with breast cancer: A systematic review commissioned by the German agency for Health Technology Assessment.

PURPOSE: To mitigate side effects of conventional cancer care and improve quality of life, many patients with breast cancer in German-speaking countries opt for mistletoe therapy in addition to standard treatment. To understand the value for users, we evaluated the domain "Patient and Social aspects" in a health technology assessment on complementary mistletoe therapy in patients with breast cancer. METHODS: A systematic review was conducted according to PRISMA guidelines. Fifteen electronic databases and the internet were searched. Qualitative studies were analyzed using qualitative content analysis; quantitative studies were systematically summarized in evidence tables. RESULTS: Seventeen studies out of 1203 screened publications with 4765 patients and 869 healthcare professionals were included in the review. The median proportion of patients using mistletoe therapy was 26.7% (range: 7.3-46.3%). Predictors for use were younger age and higher educational level. Main reasons for patients' use of mistletoe therapy were to leave nothing untried and to be actively involved in the treatment process. Reasons against use were related to a lack of knowledge or uncertainty regarding effectiveness and safety. Physicians mainly aimed to support the patient's physical condition as main reason for use and a lack of resources and knowledge as main reason against use. CONCLUSION: Mistletoe therapy was commonly used in the treatment of breast cancer despite a lack of scientific knowledge among patients and physicians. Transparent communication on motivation for use and its potential effect enables realistic expectations. Relatively small samples of mistletoe therapy users limit the representativeness and validity of our results.

[Testicular Cancer Screening in Men Aged 16 Years and Older: IQWiG ThemenCheck Health Technology Assessment Report on Medical, Economic, Social, Ethical, Legal and Organisational Aspects]. / Hodenkrebsscreening bei Männern ab 16 Jahren: IQWiG ThemenCheck Health Technology Assessment-Bericht zu medizinischen, ökonomischen, sozialen, ethischen, rechtlichen und organisatorischen Aspekten.

BACKGROUND: Testicular cancer occurs mainly in young men between 25 and 45 years and is the most common cancer at this age. Possible testicular cancer early detection measures, clinical palpation and scrotal ultrasound (CUS) or testicular self-examination (TSE) in asymptomatic men aged 16 years and older, could perhaps avoid deaths and aggressive late therapies. Therefore, we investigated whether these measures have an additional benefit compared to the current situation. Ethical, legal, social and organisational aspects were considered as well. METHODS: The methodology of this review follows IQWiG's "Allgemeine[n] Methoden Version 5.0". In addition, to estimate the theoretically possible benefits and potential harms of screening, a supplementary presentation was used for the benefit assessment based on published data from tumour registries and data on predictive values from diagnostic studies. RESULTS: No intervention studies were identified, therefore evidence-based statements on additional benefit or harm of the studied interventions could not be made. The epidemiological data showed that per 100,000 men participating in screening annually, a maximum of 1.2 advanced tumours and 0.4 deaths would have been preventable. Harm calculations suggest that with CUS of 100,000 men, 1 to 22 unnecessary testicular exposures or removals might be expected, with TSE it would be 2 cases. However, these data on the possible harm of screening are subject to great uncertainty. CONCLUSIONS: There are no intervention studies demonstrating that the benefit of testicular cancer screening in men aged 16 years and older outweighs the harm. The maximum possible additional benefit is low and chances of detection and cure are good even without screening. At present, testicular cancer screening cannot be recommended.

Additional treatment with mistletoe extracts for patients with breast cancer compared to conventional cancer therapy alone - efficacy and safety, costs and cost-effectiveness, patients and social aspects, and ethical assessment.

Background: Chemotherapy is often used in the treatment of breast cancer in women. Side effects such as diarrhea, fatigue, hair loss, fever or disturbances in blood formation impair the women's quality of life. An essential treatment goal of the accompanying mistletoe therapy (MT) used in complementary medicine is to improve the health-related quality of life during cancer therapy. Aim and methods: The HTA report on which this article is based examines the medical efficacy and safety, costs and cost-effectiveness, patient and social aspects, and ethical aspects of MT in women with breast cancer. Systematic reviews were conducted for this purpose. The search period of the literature search ranged from 2004 to October 2020. Results: A total of 2 evidence-based medical guidelines, 3 randomized trials assessing efficacy and 1 additional non-randomized intervention trial, as well as 3 observational studies assessing safety, a cost analysis, 12 cross-sectional studies on patient aspects and 17 articles on ethical evaluation were included. Improvements in health-related quality of life compared to the control group were small to moderate. Due to the high risk of bias in the studies, it is possible that the difference is not caused by MT. One study with a small sample size showed no effect on progression-free survival after 5 years. Studies on the effect of MT on overall survival are lacking. In seven studies, local skin reactions of low and moderate severity were reported in a median of 25% (range 5 to 94%) of patients, and mild to moderate systemic reactions in a median of 2% (range 0 to 8%) of patients. A comparative cost analysis from Germany reported significantly lower medical costs within 5 years after surgery for patients with MT than for patients without MT, but the underlying observational study did not control for systematic bias. With regard to patient aspects, the frequency of use and the reasons for use from the patient's or practitioner's point of view were mainly investigated. A median of 25% (range 7 to 46%) of patients with breast cancer and 29% (range 29 to 79%) of treatment providers use MT. The main motivations of patients for use were to reduce side effects, strengthen the immune system and take an active role in the treatment process. Patients felt insufficiently advised. Studies on other aspects are lacking. The ethical evaluation was able to identify 6 overarching themes; the central challenge is the insufficient evidence on efficacy and safety.

Decision-analytic modeling for early health technology assessment of medical devices - a scoping review.

Objective: The goal of this review was to identify decision-analytic modeling studies in early health technology assessments (HTA) of high-risk medical devices, published over the last three years, and to provide a systematic overview of model purposes and characteristics. Additionally, the aim was to describe recent developments in modeling techniques. Methods: For this scoping review, we performed a systematic literature search in PubMed and Embase including studies published in English or German. The search code consisted of terms describing early health technology assessment and terms for decision-analytic models. In abstract and full-text screening, studies were excluded that were not modeling studies for a high-risk medical device or an in-vitro diagnostic test. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was used to report on the search and exclusion of studies. For all included studies, study purpose, framework and model characteristics were extracted and reported in systematic evidence tables and a narrative summary. Results: Out of 206 identified studies, 19 studies were included in the review. Studies were either conducted for hypothetical devices or for existing devices after they were already available on the market. No study extrapolated technical data from early development stages to estimate potential value of devices in development. All studies except one included cost as an outcome. Two studies were budget impact analyses. Most studies aimed at adoption and reimbursement decisions. The majority of studies were on in-vitro diagnostic tests for personalized and targeted medicine. A timed automata model, to our knowledge a model type new to HTA, was tested by one study. It describes the agents in a clinical pathway in separate models and, by allowing for interaction between the models, can reflect complex individual clinical pathways and dynamic system interactions. Not all sources of uncertainty for in-vitro tests were explicitly modeled. Elicitation of expert knowledge and judgement was used for substitution of missing empirical data. Analysis of uncertainty was the most valuable strength of decision-analytic models in early HTA, but no model applied sensitivity analysis to optimize the test positivity cutoff with regard to the benefit-harm balance or cost-effectiveness. Value-of-information analysis was rarely performed. No information was found on the use of causal inference methods for estimation of effect parameters from observational data. Conclusion: Our review provides an overview of the purposes and model characteristics of nineteen recent early evaluation studies on medical devices. The review shows the growing importance of personalized interventions and confirms previously published recommendations for careful modeling of uncertainties surrounding diagnostic devices and for increased use of value-of-information analysis. Timed automata may be a model type worth exploring further in HTA. In addition, we recommend to extend the application of sensitivity analysis to optimize positivity criteria for in-vitro tests with regard to benefit-harm or cost-effectiveness. We emphasize the importance of causal inference methods when estimating effect parameters from observational data.

Causal evidence in health decision making: methodological approaches of causal inference and health decision science.

Objectives: Public health decision making is a complex process based on thorough and comprehensive health technology assessments involving the comparison of different strategies, values and tradeoffs under uncertainty. This process must be based on best available evidence and plausible assumptions. Causal inference and health decision science are two methodological approaches providing information to help guide decision making in health care. Both approaches are quantitative methods that use statistical and modeling techniques and simplifying assumptions to mimic the complexity of the real world. We intend to review and lay out both disciplines with their aims, strengths and limitations based on a combination of textbook knowledge and expert experience. Methods: To help understanding and differentiating the methodological approaches of causal inference and health decision science, we reviewed both methods with the focus on aims, research questions, methods, assumptions, limitations and challenges, and software. For each methodological approach, we established a group of four experts from our own working group to carefully review and summarize each method, followed by structured discussion rounds and written reviews, in which the experts from all disciplines including HTA and medicine were involved. The entire expert group discussed objectives, strengths and limitations of both methodological areas, and potential synergies. Finally, we derived recommendations for further research and provide a brief outlook on future trends. Results: Causal inference methods aim for drawing causal conclusions from empirical data on the relationship of pre-specified interventions on a specific target outcome and apply a counterfactual framework and statistical techniques to derive causal effects of exposures or interventions from these data. Causal inference is based on a causal diagram, more specifically, a directed acyclic graph (DAG), which encodes the assumptions regarding the causal relations between variables. Depending on the type of confounding and selection bias, traditional statistical methods or more complex g-methods are needed to derive valid causal effects. Besides the correct specification of the DAG and the statistical model, assumptions such as consistency, positivity, and exchangeability must be checked when aiming at causal inference. Health decision science aims for guiding policy decision making regarding health interventions considering and balancing multiple competing objectives of a decision based on data from multiple sources and studies, for example prevalence studies, clinical trials and long-term observational routine effectiveness studies, and studies on preferences and costs. It involves decision analysis, a systematic, explicit and quantitative framework to guide decisions under uncertainty. Decision analyses are based on decision-analytic models to mimic the course of disease as well as aspects and consequences of the intervention in order to quantitatively optimize the decision. Depending on the type of decision problem, decision trees, state-transition models, discrete event simulation models, dynamic transmission models, or other model types are applied. Models must be validated against observed data, and comprehensive sensitivity analyses must be performed to assess uncertainty. Besides the appropriate choice of the model type and the valid specification of the model structure, it must be checked if input parameters of effects can be interpreted as causal parameters in the model. Otherwise results will be biased. Conclusions: Both causal inference and health decision science aim for providing best causal evidence for informed health decision making. The strengths and limitations of both methods differ and a good understanding of both methods is essential for correct application but also for correct interpretation of findings from the described methods. Importantly, decision-analytic modeling should be combined with causal inference when developing guidance and recommendations regarding decisions on health care interventions.

A universal outcome measure for headache treatments, care-delivery systems and economic analysis.

BACKGROUND: The first manuscript in this series delineated a model of structured headache services, potentially cost-effective but requiring formal cost-effectiveness analysis (CEA). We envisaged a need for a new outcome measure for this purpose, applicable to all forms of treatment, care and care-delivery systems as opposed to comparisons of single-modality treatments. CONCEPTION AND DELINEATION: A literature review confirmed the lack of any suitable established measure. We prioritised construct validity, simplicity, comprehensiveness and expression in intuitive units. We noted that pain was the key burdensome symptom of migraine and episodic tension-type headache (TTH), that pain above a certain level was disabling, that it was difficult to put economic value to pain but relatively easy to do this for time, a casualty of headache leading to lost productivity. Alleviation of pain to a non-disabling level would be expected to bring restoration of function. We therefore based the measure on time spent in the ictal state (TIS) of migraine or TTH, either as total TIS or proportion of all time. We expressed impact on health, in units of time, as TIS*DW, where DW was the disability weight for the ictal state supplied by the Global Burden of Disease (GBD) studies. If the time unit was hours, TIS*DW yielded hours lived with (or lost to) disability (HLDs), in analogy with GBD's years lived with disability (YLDs). UTILITY ASSESSMENT: Acute treatments would reduce TIS by shortening attack duration, preventative treatments by reducing attack frequency; health-care systems such as structured headache services would have these effects by delivering these treatments. These benefits were all measurable as HLDs-averted. Population-level estimates would be derived by factoring in prevalence, but also taking treatment coverage and adherence into account. For health-care systems, additional gains from provider-training (promoting adherence to guidelines and, therefore, enhancing coverage) and consumer-education (improving adherence to care plans), increasing numbers within populations gaining the benefits of treatments, would be measurable by the same metric. CONCLUSIONS: The new outcome measure expressed in intuitive units of time is applicable to treatments of all modalities and to system-level interventions for multiple headache types, with utility for CEA and for informing health policy.

Adult patient access to electronic health records.

BACKGROUND: To support patient-centred care, healthcare organisations increasingly offer patients access to data stored in the institutional electronic health record (EHR). OBJECTIVES: Primary objective 1. To assess the effects of providing adult patients with access to electronic health records (EHRs) alone or with additional functionalities on a range of patient, patient-provider, and health resource consumption outcomes, including patient knowledge and understanding, patient empowerment, patient adherence, patient satisfaction with care, adverse events, health-related quality of life, health-related outcomes, psychosocial health outcomes, health resource consumption, and patient-provider communication. Secondary objective 1. To assess whether effects of providing adult patients with EHR access alone versus EHR access with additional functionalities differ among patient groups according to age, educational level, or different status of disease (chronic or acute). SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, PsycINFO, CINAHL, and Scopus in June 2017 and in April 2020. SELECTION CRITERIA: Randomised controlled trials and cluster-randomised trials of EHR access with or without additional functionalities for adults with any medical condition. DATA COLLECTION AND ANALYSIS: We used standard Cochrane methodological procedures. MAIN RESULTS: We included 10 studies with 78 to 4500 participants and follow-up from 3 to 24 months. Nine studies assessed the effects of EHR with additional functionalities, each addressing a subset of outcomes sought by this review. Five studies focused on patients with diabetes mellitus, four on patients with specific diseases, and one on all patients. All studies compared EHR access alone or with additional functionalities plus usual care versus usual care only. No studies assessing the effects of EHR access alone versus EHR access with additional functionalities were identified. Interventions required a variety of data within the EHR, such as patient history, problem list, medication, allergies, and lab results. In addition to EHR access, eight studies allowed patients to share self-documented data, seven offered individualised disease management functions, seven offered educational disease-related information, six supported secure communication, and one offered preventive reminders. Only two studies were at low or unclear risk of bias across domains. Meta-analysis could not be performed, as participants, interventions, and outcomes were too heterogeneous, and most studies presented results based on different adjustment methods or variables. The quality of evidence was rated as low or very low across outcomes. Overall differences between intervention and control groups, if any, were small. The relevance of any small effects remains unclear for most outcomes because in most cases, trial authors did not define a minimal clinically important difference. Overall, results suggest that the effects of EHR access alone and with additional functionalities are mostly uncertain when compared with usual care. Patient knowledge and understanding: very low-quality evidence is available from one study, so we are uncertain about effects of the intervention on patient knowledge about diabetes and blood glucose testing. Patient empowerment: low-quality evidence from three studies suggests that the intervention may have little or no effect on patient empowerment measures. Patient adherence: low-quality evidence from two studies suggests that the intervention may slightly improve adherence to the process of monitoring risk factors and preventive services. Effects on medication adherence are conflicting in two studies; this may or may not improve to a clinically relevant degree. Patient satisfaction with care: low-quality evidence from three studies suggests that the intervention may have little or no effect on patient satisfaction, with conflicting results. Adverse events: two small studies reported on mortality; one of these also reported on serious and other adverse events, but sample sizes were too small for small differences to be detected. Therefore, low-quality evidence suggests that the intervention may have little to no effect on mortality and other adverse events. Health-related quality of life: only very low-quality evidence from one study is available. We are uncertain whether the intervention improves disease-specific quality of life of patients with asthma. Health-related outcomes: low-quality evidence from eight studies suggests that the intervention may have little to no effect on asthma control, glycosylated haemoglobin (HbA1c) levels, blood pressure, low-density lipoprotein or total cholesterol levels, body mass index or weight, or 10-year Framingham risk scores. Low-quality evidence from one study suggests that the composite scores of risk factors for diabetes mellitus may improve slightly with the intervention, but there is uncertainty about effects on ophthalmic medications or intraocular pressure. Psychosocial health outcomes: no study investigated psychosocial health outcomes in a more than anecdotal way. Health resource consumption: low-quality evidence for adult patients in three studies suggests that there may be little to no effect of the intervention on different measures of healthcare use. Patient-provider communication: very low-quality evidence is available from a single small study, and we are uncertain whether the intervention improves communication measures, such as the number of messages sent. AUTHORS' CONCLUSIONS: The effects of EHR access with additional functionalities in comparison with usual care for the most part are uncertain. Only adherence to the process of monitoring risk factors and providing preventive services as well as a composite score of risk factors for diabetes mellitus may improve slightly with EHR access with additional functionalities. Due to inconsistent terminology in this area, our search may have missed relevant studies. As the overall quality of evidence is very low to low, future research is likely to change these results. Further trials should investigate the impact of EHR access in a broader range of countries and clinical settings, including more patients over a longer period of follow-up, as this may increase the likelihood of detecting effects of the intervention, should these exist. More studies should focus on assessing outcomes such as patient empowerment and behavioural outcomes, rather than concentrating on health-related outcomes alone. Future studies should distinguish between effects of EHR access only and effects of additional functionalities, and investigate the impact of mobile EHR tools. Future studies should include information on usage patterns, and consider the potential for widening health inequalities with implementation of EHR access. A taxonomy for EHR access and additional functionalities should be developed to promote consistency and comparability of outcome measures, and facilitate future reviews by better enabling cross-study comparisons.

Effects of Adult Patient Portals on Patient Empowerment and Health-Related Outcomes: A Systematic Review.

Patient portals are offered by health care organizations to facilitate health information sharing and patient empowerment and support patient-centered care. The aim of this systematic review is to assess the effect of patient portals on patient empowerment and health-related outcomes. After a systematic literature search, ten randomized controlled trials (RCTs) were included in this review. Of these, seven RCTs were conducted in the United States., two in Canada, and one in Japan. Study characteristics, risk of bias, and outcomes were extracted. varied in terms of intervention, included patients, and outcome. Most studies found no or only a small, clinically non-relevant effect of patient portals. The review showed that future research should develop a taxonomy to describe patient portal functionalities to facilitate the aggregation of evidence.

Recommendations for primary studies evaluating therapeutic medical devices were identified and systematically reported through reviewing existing guidance.

OBJECTIVES: The aim of the study was to review existing recommendations on study design, conduct, analysis, and reporting for primary studies of therapeutic medical devices (TMDs) and the closely related field of interventional procedures. STUDY DESIGN AND SETTING: We performed a targeted literature review of publications with recommendations for study design, conduct, analysis, and reporting for primary studies of TMDs and related technologies. We combined an electronic database search with a systematic screening of tables of content of selected journals and scanning the reference lists of relevant articles. RESULTS: We identified 40 publications authored or commissioned primarily by regulators, health technology assessment agencies, and expert groups. We identified study designs of randomized clinical trials that specifically address the quick, incremental development of TMDs and provider and patient preferences. The importance of contextual factors for TMD interventions should be considered during the selection of patients, providers, and centers, as well as in data collection and analysis. We also identified guidance for the analysis and quantification of learning curves as well as for the design and analysis of large registries of high quality. CONCLUSION: The methodology to conduct primary research for TMDs should be disseminated to support improvement of the evidence base for health technology assessments.

Ten recommendations for assessing the comparative effectiveness of therapeutic medical devices: a targeted review and adaptation.

OBJECTIVES: The aim of this study was to review existing guidance on the methods for evaluation of the comparative effectiveness (CE) of therapeutic medical devices (TMDs) and develop recommendations for systematic reviews (SRs) of CE of TMDs as part of health technology assessments. STUDY DESIGN AND SETTING: We performed a targeted literature review of evaluation frameworks and recommendations for evaluation methods for TMDs and related technologies. An electronic database search was combined with systematic screening of tables of content of selected journals and scanning the reference lists of relevant papers. RESULTS: We identified a framework for SRs of complex interventions (FSRCIs) and three documents providing detailed methodology for TMD evaluation. We applied the FSRCIs to TMD evaluation taking into account the more complex composition of the intervention, its rapid, incremental development, as well as the dependency of treatment effects on contextual factors and user proficiency. In 10 recommendations, including a template for a logic model for TMD, we summarized relevant factors that should be systematically considered. CONCLUSION: The quality of SRs of CE assessments of TMDs can be improved by applying these recommendations. This also may require an increase of the quality of primary studies.

INTEGRATE-HTA: THE PERSPECTIVE OF EUnetHTA.

The HTA Core Model (HTACM) of EUnetHTA and the INTEGRATE-HTA Model (IHTAM) both provide HTA experts with advanced guidance on how to assess health technologies. In this study, we examine the similarities and differences of the two models, identifying synergies and opportunities for future collaboration. We also consider how such an alignment of the HTACM and IHTAM might be done in practice and present some alternative practical approaches. Overall the two models share several similarities, primarily the perception of HTA as a multidisciplinary analysis that needs to be adjusted according to the properties of the technology under assessment.

A bias-adjusted evidence synthesis of RCT and observational data: the case of total hip replacement.

Evaluation of clinical effectiveness of medical devices differs in some aspects from the evaluation of pharmaceuticals. One of the main challenges identified is lack of robust evidence and a will to make use of experimental and observational studies (OSs) in quantitative evidence synthesis accounting for internal and external biases. Using a case study of total hip replacement to compare the risk of revision of cemented and uncemented implant fixation modalities, we pooled treatment effect estimates from OS and RCTs, and simplified existing methods for bias-adjusted evidence synthesis to enhance practical application. We performed an elicitation exercise using methodological and clinical experts to determine the strength of beliefs about the magnitude of internal and external bias affecting estimates of treatment effect. We incorporated the bias-adjusted treatment effects into a generalized evidence synthesis, calculating both frequentist and Bayesian statistical models. We estimated relative risks as summary effect estimates with 95% confidence/credibility intervals to capture uncertainty. When we compared alternative approaches to synthesizing evidence, we found that the pooled effect size strongly depended on the inclusion of observational data as well as on the use bias-adjusted estimates. We demonstrated the feasibility of using observational studies in meta-analyses to complement RCTs and incorporate evidence from a wider spectrum of clinically relevant studies and healthcare settings. To ensure internal validity, OS data require sufficient correction for confounding and selection bias, either through study design and primary analysis, or by applying post-hoc bias adjustments to the results. © 2017 The Authors. Health Economics published by John Wiley & Sons, Ltd.

USING EXPERT OPINION IN HEALTH TECHNOLOGY ASSESSMENT: A GUIDELINE REVIEW.

OBJECTIVES: External experts can be consulted at different stages of an HTA. When using vague information sources, it is particularly important to plan, analyze, and report the information processing in a standardized and transparent way. Our objective was to search and analyze recommendations regarding where and how to include expert data in HTA. METHODS: We performed a systematic database search and screened the Internet pages of seventy-seven HTA organizations for guidelines, recommendations, and methods papers that address the inclusion of experts in HTA. Relevant documents were downloaded, and information was extracted in a standard form. Results were merged in tables and narrative evidence synthesis. RESULTS: From twenty-two HTA organizations, we included forty-two documents that consider the use of expert opinion in HTA. Nearly all documents mention experts in the step of preparation of the evidence report. Six documents address their role for priority setting of topics, fifteen for scoping, twelve for the appraisal of evidence and results, another twelve documents mention experts when considering the dissemination of HTA results. During the assessment step, experts are most often asked to amend the literature search or to provide expertise for special data analyses. Another issue for external experts is to appraise the HTA results and refer them back to a clinical and social context. Little is reported on methods of expert elicitation when their input substitutes study data. CONCLUSIONS: Despite existing recommendations on the use of expert opinion in HTA, common standards for elicitation are scarce in HTA guidelines.

Clinical effectiveness of cancer screening biomarker tests offered as self-pay health service: a systematic review.

BACKGROUND: Biomarker tests are increasingly being offered by laboratories and clinicians as self-pay health services to screen asymptomatic individuals; however, sufficient evidence may not be available to support this practice. We investigated the benefit-harm tradeoffs associated with 11 biomarkers currently offered in Germany as self-pay tests to screen for cancer. METHODS: We systematically searched bibliographic databases for health technology assessments, systematic reviews and randomized-controlled trials (RCTs) through September 2015. We included publications that analysed cancer screening biomarkers and reported patient-relevant outcomes (mortality, morbidity, quality of life), and potential harms of screening, among asymptomatic individuals in screening and non-screening arms. Language was restricted to English and German. Two reviewers independently screened references; data were extracted and quality of included studies was evaluated by a reviewer and validated by a second reviewer. RESULTS: Six publications of secondary literature and four publications reporting results from two RCTs were included. For 10 cancer screening biomarkers, no direct evidence on patient-relevant outcomes was available. Only one trial, which simultaneously assessed cancer antigen 125 (CA125) and vaginal ultrasound for ovarian cancer screening, provided the outcome of interest. Screening compared with usual care did not reduce ovarian cancer mortality. Patient harms included overdiagnosis and false-positive results. CONCLUSION: Although ovarian cancer screening with CA125 showed no benefit, false-positive tests, overdiagnosis and overtreatment were reported. Physicians and laboratories should provide patients with comprehensive information about the lack of evidence and potential harms caused by biomarker screening tests offered as a self-pay health service.

Systematic overview of cost-effectiveness thresholds in ten countries across four continents.

AIM: To provide an overview of thresholds for incremental cost-effectiveness ratios (ICERs) representing willingness-to-pay (WTP) across multiple countries and insights into exemptions pertaining to the ICER (e.g., cancer). To compare ICER thresholds to individual country's estimated ability-to-pay. MATERIALS & METHODS: We included AHRQ/USA, BIQG-GOEG/Austria, CADTH/Canada, DAHTA@DIMDI/Germany, DECIT-CGATS/Brazil, HAS/France, HITAP/Thailand, IQWiG/Germany, LBI-HTA/Austria, MSAC/Australia, NICE/England/Wales and SBU/Sweden. ICER thresholds were derived from systematic literature/website search/expert surveys. WTP was compared with ATP using Spearman's rank correlation. RESULTS: Two general and explicitly acknowledged thresholds (England/Wales, Thailand), implicit thresholds in six countries and different ICER thresholds/decision-making rules in oncology were identified. Correlation between WTP and ability-to-pay was moderate. DISCUSSION: Our overview supports country-specific discussions on WTP and on how to define value(s) within societies.

[HTA goes Europe: European collaboration on joint assessment and methodological issues becomes reality]. / HTA goes Europe: Europäische Zusammenarbeit in gemeinsamen Nutzenbewertungen und methodischen Fragen wird real.

INTRODUCTION: The standardisation of European HTA and thus the reduction of redundancies require clearly defined processes and methods. The HTA Core Model®, a tool developed by the European Network EUnetHTA, is intended to ensure the transparent production of standardised and high-quality assessments in international collaboration. METHODS: The present paper describes the experience with already published EUnetHTA assessments as well as possibilities for national/local adaptations of these assessments. The integration of jointly developed methods in routine processes of individual HTA agencies will be explained on the basis of a selected example. Further methodological initiatives in Europe will be presented. RESULTS: So far, EUnetHTA has published four rapid assessments conducted through European cooperation between 6-9 HTA institutes during Joint Action 2 (2012-2015). Two assessments dealt with pharmaceuticals and two with non-pharmaceutical interventions. The overall duration of these assessments ranged from 7 to 9 months. There is initial information about the frequency and manner in which these assessments have been used for national/local HTA reports. According to a survey, a total of 28 HTA institutes have indicated that they want to make use of these assessments in their own context. In Austria, the Ludwig Boltzmann Institute for Health Technology Assessment (LBI-HTA) has produced two reports based on EUnetHTA assessments. A further step towards cross-border collaboration and harmonisation is the implementation of these tools in a national and regional context. Beginning in 2015 the LBI-HTA will adjust two programme lines to the format of the HTA Core Model® in order to increase the transferability of HTAs and to reduce redundancies. DISCUSSION: Barriers to European collaboration include the relevance of topics for individual HTA institutes and the timing of joint assessments. Implementing commonly developed methods as standard practice in local/national HTA institutes is mainly impeded by legislative requirements. CONCLUSION: Despite the initial positive experiences with international collaboration on specific topics and methods, the coming years will have to prove whether existing barriers can be overcome effectively.

Health technology assessment of medical devices: What is different? An overview of three European projects.

BACKGROUND: With the growing use and importance of health technology assessment (HTA) in decision making during recent years, health technology assessors, decision makers and stakeholders are confronted with methodological challenges due to specific characteristics of health technologies (e. g., pharmaceuticals, diagnostic tests, screening programs), their developmental environment, and their regulation process. Being aware of the necessity to use HTA as a policy instrument for sustainable health care systems in a regulatory environment of decentralized Conformité Européenne (CE) marking, the European Union (EU) is increasingly supporting the development of methods for the assessment of medical devices (MD) on different levels: within the scope of European research projects and within joint assessment activities of the member states of the European network for Health Technology Assessment (EUnetHTA). OBJECTIVE: First, this article describes three projects: MedtecHTA-Methods for Health Technology Assessment of Medical Devices, a European Perspective Work Package 3 (WP3), Comparative Effectiveness of Medical Devices led by the University for Health Sciences, Medical Informatics and Technology (UMIT). Second, we discuss the experiences of the Ludwig Boltzmann Institute Health Technology Assessment (LBI HTA) with the joint production of rapid assessments of medical devices by several European HTA agencies within EUnetHTA. Third, a brief outline is given of the framework of joint methodological guideline elaboration by the EUnetHTA partner organizations because a guideline for therapeutic MD is also being developed here. METHODS: We will describe aims, methods and some preliminary results of MedtecHTA and EUnetHTA Joint Action 2 Work Package 5 Strand B (WP5B) applying the HTA Core Model for Rapid Assessment for national adaptation and reporting, and give an overview of the development process of methodological guidelines within WP 7 of EUnetHTA Joint Action 2. RESULTS: Based on a literature review in MedtecHTA WP3 incremental development, context dependency and the physical mode of action of MD were identified as those characteristics making therapeutic MD different from drugs with regard to evaluation methods. In addition, regulation does not stipulate clinical trials. These characteristics were also identified as challenges for the production of joint assessments of MD within the HTA network EUnetHTA. Furthermore, adequate timing of assessment production, the variety of involved manufacturers, the non-transparent regulation process of MD in Europe and the often poor evidence base pose a challenge to EUnetHTA assessors. As a consequence, processes and methods for the joint production of rapid assessments must be continuously adapted and improved. DISCUSSION: Research on HTA methods for the assessment of MD tries to provide tools to deal with rapidly developing devices during evidence generation, dependence of clinical effectiveness of MD on user experience and context factors. There are also tools to integrate evidence from different sources adjusting for different levels of validity, but these methods are not established and need high epidemiological and statistical expertise. A framework for deciding whether additional evidence is needed to reduce uncertainty regarding safety, clinical effectiveness and cost-effectiveness will be adapted to MD. The whole process of evidence generation before and after market access has to be considered to provide an environment for conclusive HTA recommendations informing health care decision making. In Joint Action 2, EUnetHTA develops transparent processes for the early dialogue with stakeholders and fosters dissemination of appropriate HTA methods. In the case of MD, there are special accumulated needs for such efforts.

Oncotyrol--Center for Personalized Cancer Medicine: Methods and Applications of Health Technology Assessment and Outcomes Research.

BACKGROUND: The Oncotyrol - Center for Personalized Cancer Medicine is an international and interdisciplinary alliance combining research and commercial competencies to accelerate the development, evaluation and translation of personalized healthcare strategies in cancer. The philosophy of Oncotyrol is to collaborate with relevant stakeholders and advance knowledge "from bench to bedside to population and back". Oncotyrol is funded through the COMET Excellence Program by the Austrian government via the national Austrian Research Promotion Agency (FFG). This article focuses on the role of health technology assessment (HTA) and outcomes research in personalized cancer medicine in the context of Oncotyrol. METHODS: Oncotyrol, which currently comprises approximately 20 individual projects, has four research areas: Area 1: Biomarker and Drug Target Identification; Area 2: Assay Development and Drug Screening; Area 3: Innovative Therapies; Area 4: Health Technology Assessment and Bioinformatics. Area 4 translates the results from Areas 1 to 3 to populations and society and reports them back to Area 3 to inform clinical studies and guidelines, and to Areas 1 and 2 to guide further research and development. RESULTS: In a series of international expert workshops, the Oncotyrol International Expert Task Force for Personalized Cancer Medicine developed the Methodological Framework for Early Health Technology Assessment and Decision Modeling in Cancer and practical guidelines in this field. Further projects included applications in the fields of sequential treatment of patients with chronic myeloid leukemia (CML), benefit-harm and cost-effectiveness evaluation of prostate cancer screening, effectiveness and cost-effectiveness of multiple cervical cancer screening strategies, and benefits and cost-effectiveness of genomic test-based treatment strategies in breast cancer. CONCLUSION: An interdisciplinary setting as generated in Oncotyrol provides unique opportunities such as systematically coordinating lab and bench research, product development, clinical studies and decision science/HTA and transparent joint planning of research and development with a partnership of researchers, manufacturers and health policy decision makers. However, generating a joint research and legal framework with numerous partners from different sectors can be challenging, particularly in the starting period of such an endeavor. The journey to translational personalized medicine through multidisciplinary collaborations may still be long and difficult, but it is evident that it must be continued to turn vision into reality.